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(57) Abstract 

A camera assembly for use in an eye tracking apparatus, the camera including a camera with a lens having an image axis; and a 
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AN EYE TRACKER -USING AN OFF-AXIS, 
RING ILLUMINATION SOURCE 

Background of the Invention 
5 Generally, the invention relates to an eye tracker 

for determining line-of-gaze of a subject and, more 
particularly, it relates to an eye tracker that uses an 
improved light source for illuminating the subject's eye. 
All practical eye movement measurement or eye 

10 tracking techniques involve tracking one or more features 
or reflections that can be optically detected on the eye. 
Many of the system that are available fall into one of 
two categories, namely, methods that detect single 
features on the subject's eye and methods that detect two 

15 features on the subject's eye. One way of establishing a 
feature is by reflecting a. light source off of the eye. 
One feature that has been often used for this purpose is 
the pupil. In such systems, the equipment determines the 
position of the pupil center. Through a simple 

2 0 mathematical transformation, changes in the position of 
the center of the pupil can be easily converted to an 
indication of the line-of-gaze of the subject. 

There are many known algorithms for finding the 
pupil center. The most appropriate algorithms depend on 

2 5. the type of sensor being used, the desired measurement 

update rate, and the amount of computer processing power 
that is available in the equipment . 

The retina is highly reflective, but any light 
reflected back through the pupil will be directed towards 
30 its original source. In fact, if the eye is focused at 
the plane of the source, such retro-reflected light from 
the retina will be imaged back- at the source. Under 
normal viewing conditions, the retina looks completely 
black, because none of the rays reflected . of f of the 

3 5 retina return to the observer. If, however, the observer 
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is able to look along the axis of an illumination beam, 
then the observer will see the retinal reflection and the 
pupil will appear bright. 

Many currently available commercial eye tracker 
5 systems use the backlit "bright" pupil effect to perform 
the eye tracking. This is because. the bright pupil tends 
to be easier to recognize than a dark pupil, especially 
under low light conditions or if the surrounding features 
are dark. Also, the bright pupil contrast tends to 
10 increase as the detector to the eye distance increase. 

Thus, systems which use the bright pupil effect tend to. 
operate better with the detector farther from the eye 
than do systems which use the dark pupil. 

It is commonly believed by persons skilled in the 
15 art that bright pupil contrast can be adversely affected 
by any illumination that is hot coaxial with the camera 
or detector. Thus, systems which use the bright pupil 
technique are designed to keep the illuminating light on 
the viewing axis. One common approach to achieving this 
20 is to use a beam splitter, which typically is a prism 

with a 45° reflecting surface. The viewing camera looks 
through the beam splitter (and through the reflecting 
surface) at the subject's eye. The illumination source, 
which is off to the side of the viewing axis, directs 
25 light at the 45° reflecting surface of the beam splitter 
■ which reflects that , light along and coaxial . with the 
viewing axis toward the subject. In such 
implementations, the beam splitter adds to the bulk of 
the device it tends to attenuate the light reflected back 
3 0 from the subject's eye to the camera. 

Another approach is to place the light source in 
the middle of the viewing lens. This guarantees that the 
light is coaxial with the viewing axis, but it also 
blocks part of the lens and thus reduces its efficiency.. 



ri 
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Summary of the Invention 
In general, in one aspect, the invention is a 
. camera unit for use in an eye tracking apparatus. The 
camera unit includes a camera with a lens having ah image 
5 axis; and a ring shaped light source disposed around the 
image axis and near the periphery of the lens aperture. 
The light source is oriented to direct light along the 
camera axis toward the target. 

In general, in another aspect, the invention is an 
10 eye line-of -gaze measurement apparatus including an 
electronic camera with a lens having an image axis; a 
ring shaped light source disposed around the image axis 
and near the periphery of the lens aperture and oriented 
. to direct light along the camera axis toward the target; 
15 and a digital processor programmed to. determine an eye 
line-of -gaze from the image of a ret ro- reflect ion 
obtained from the subject's eye. 

Preferred embodiments include the following 
features. The the ring-shaped light source includes an 
20 array of lights arranged in a circle to form a ring. The 
plurality of light sources are evenly spaced about the 
circle. The light sources are LED's. 

In general, in still another aspect, the invention 
is a method of generating a retro-reflection from a 
25 subject's eye for use in a line-of -gaze measurement 

system that utilizes a bright pupil detection technique. 
The method includes the steps of producing an image of 
the subject's eye by using a camera that is characterized 
by a viewing axis; and illuminating the subject's eye 
3 0 with an off -axis illumination to produce a retro- 
reflection from the retina of the subject's eye. 

In preferred embodiments, the step of illuminating 
the subject's eye involves illuminating with a light 
source that is distributed around the viewing axis. 
35 One advantage of the invention is that even though 
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the illumination is placed off axis . from the viewing 
camera it nevertheless produces a surprisingly effective 
- retro-reflection from the eyes of target subjects. 

Another advantage of the invention is that it 
5 provides a larger and brighter illumination source which 
both improves the accuracy of the eye tracking system and 
makes the overall system less sensitive to variations in. 
ambient light conditions. The improved light source more 
effectively produces a retro-reflection from the retina 
10 of the subject's eye. Also, it produces a large enough 
reflection off of the cornea and at the same time it 
produces a bright . enough retinal retro-reflection from 
which the point-of -gaze determinations can be made. 

Yet another advantage of the invention is that it 
15 eliminates the need for inserting between the viewing 

camera and the subject's eye a beam splitter which also 
acts to attenuate the intensity of the image obtained by 
the camera . 

Other advantages and features will become apparent 
2 0 from the following description of the preferred 
embodiment and from the claims. 

Brief Description of the Drawings 
Fig. 1 is a block diagram of representative 
components of an eye tracker system including a camera 
2 5 unit which has a ring illumination source;. 

Fig.. 2 is a front view of ring illumination source 
that is mounted on" the front of the camera unit shown in 
Fig. 1: 

Fig. 3 is an embodiment of an eye-head tracker 
.3 0 system including a head-mounted scene camera, a 

stationary scene camera, and a eye t acker camera which 
includes the ring illumination source; and 

Fig. 4 is an alternative embodiment of an eye -head 
tracker system including a stationary scene camera. 
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Description of the Preferred Embodiments 
Referring to Fig. 1, an eye tracker system with an 
- improved illumination source includes a solid -state 
camera 10, a ring light source 12 for illuminating the 
5 subjects eye 14 (also shown from a front perspective in 
Fig. 2), and a digital processor 16 for executing the 
programs which processes the camera image to locate the 
pupil, determine its center, and then compute, the line- 
of-gaze of . the subject. The novel aspects of the system 
10 are in the design and construction of the ring light 

source that is used to illuminate the subject's eye and 
its Relationship with the camera lens. 

The camera can be any commercially available 
system that is. suitable for generating an image of the 
::15 eye. For; example, it can include a solid state linear or 
two-dimensional array device. The arrays can be made by 
clustering photocells in. tightly packed linear or two- 
dimensional arrangements. Alternatively, they can 
include arrays of thousands of charge coupled devices 
.20 (CCD's) or charge injection devices (CID's) such as are 
found in commercially available solid state cameras or 
video cameras. In the described embodiment, the camera is 
a Sony EVI-D3 0 pan/tilt color camera, which is a compact 
solid state device and which uses a CD array. . 
25 The Sony camera that is used in the described 

embodiment includes a lens system 20 which focuses the 
. image onto a CCD array 22. It also includes internal 
electronic circuitry 24 which converts the signals from 
the CCD array into. a form that is displayable and is 
3 0 analyzable by the image processing components of the 
system. Of course, any camera that generates an image 
which can be processed and analyzed to extract feature 
information would be acceptable. 

Since the camera was operated at a distance of 
3 5 between 18 to 4 0 inches from the subjects eye, we further 
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modified the lens system within the Sony camera to 
magnify the image to a size that was more useful to the 
. image processing system. This involved adding, a positive 
lens 2 8 before internal lens system 2 0 and adding a 
5 negative lens 30 after the lens system 20. The added 
front lens was a piano convex lens obtained from Edmund 
Scientific (part.no. 44020) with a 20.5 mm diameter and a 
415 mm focal, length. the added back lens was a negative 
lens also obtained from Edmund Scientific (part no. 
10 44090) with a 9.2 mm diameter and a negative focal length 
of 41 mm. 

We also modified the Sony camera by building ah 
addition to the front of the camera which included a 
. plate 40 supporting the ring of .illumination 13 that 

15 encircles and is in close proximity to the lens opening. 
The lens system is contained in a housing 21 onto, which 
plate 4 0 is mounted. Plate 40 has a centrally located 
circular hole 45 through which the camera views the 
target scene. The hole in plate 40 is approximately the 

20 size of the aperture of the lens. More specifically, the 
hole is made as small as possible without either 
compromising the light gathering efficiency of . the lens 
or the quality of the image that is produced. 
Surrounding hole 45, there is an array of eight, evenly 

25 spaced LED' s 13 mounted on the plate and oriented to 
direct- light toward the target scene . The LED ' s have 
built in lenses which produce a narrow beam than would be 
generated by the device without the lens. The LED's 
produce light that . is in the near infra-red region, 

3 0 though of course, light of other wavelengths could also 
be used. The advantage of near infra-red light is that 
since it is not visible to people, it will not distract 
them. 

In the described embodiment, the LED's were 
35 devices sold Siemens Corporation (part no. SFH 484) which 
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produce a 16° beam at a center wavelength of about. 880 
nm. Also, in the described embodiment, the lens aperture 
. formed in the plate was about 0.5 inch and. the. diameter 
of the ring of light was slightly bigger, e.g 0.7 3 inch. 
5 We note that it is desirable to mount the ring of light 
so that its diameter is as small, i.e., it is as close as 
possible, the central viewing axis 50 (see Fig; 1) of the 
lens to maximize the retro-reflection that is obtained 
-from the. subject's eye. However, as indicated above, the 
10 size of the array should not be so small as to interfere 
with the efficient operation. of the lens. If the ring 
diameter is too large, then the light source will become 
less effective at producing the retro-reflection. 
Indeed, at a certain diameter, it will loose all ability 
-15 to produce a retro- reflection that can be observed by the 
camera . 

As we noted, the array of closely spaced LED' s 
forms a ring of. light surrounding the lens aperture. It 
should be understood, however, that any effective way of 

-20 producing a ring of light would be acceptable. For 

example, one could use a single light source and an group 
of optical fibers to produce the array of individual 
light sources. Or alternatively, one could use an 
optical lens system to produce the ring of light. It. 

25 should, also be. noted that the retro- reflect ion can be 
generated using less, than a complete ring of light. 
Indeed, . what was surprising is that it was. possible to 
effectively produce the retro-reflection with an off axis 
light source. 

30- Though we have described the. eye tracking system 

as using the pupil center to determine line-of -gaze we 
do not mean to imply that the eye tracking system is 
: limited to using only a single feature of the eye to 
compute line-pf -gaze . In fact, the system may use 

35 multiple features also including, for example ,. corneal 
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reflection of the light source. In the case of corneal 
reflection, the feature is typically at a different 
- position from the center of the pupil. 

Tracking only the position of a single landmark or 
5 feature of the eye does not permit the system to 

distinguish between eye rotation and eye translation with 
respect to the camera. Thus,, further information must be 
provided such as could be obtained from a head tracker 
that indicates the position of the head with respect to 

10 the camera. Another source of information can be a second 
feature on the eye, e.g. corneal reflection. Since the 
second feature is at a different location from the first 
feature,., the system can eliminate the ambiguity the 
ambiguity , between translation and rotation. Of course, 

15 the techniques for using two features to perform eye 
tracking are well known in the art arid will not be 
described here. But it should be understood that the 
invention is meant to cover systems and method which use 
these other techniques in addition to the bright pupil 

2 0 technique. 

Examples of two embodiments of a complete system 
which utilize double features to perform eye tracking are 
shown in Figs. 3 and 4 and will now be described. 

Fig. 3 shows a typical eye tracking system which 
25 uses the improved light source for bright pupil. 

monitoring.. In the system shown in Fig. 3, a person whose 
. point of gaze is being measured wears a helmet 110 on 
which are mounted a visor 112, an eye tracker sensor and 
optics unit 114, having the features described above, and 

3 0 a head mounted .scene camera 116. Visor 112 is coated to 

be very reflective in the near infra-red but transparent 
in the visual spectrum and thus allows the person to look 
through it while at the same time eye tracker sensor and 
optics unit 114 is able to "look" at a reflection of the 
3 5 person's eye and head-mounted scene camera 116 is able to 
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see a reflection of the field of view of the subject. A 
stationary scene camera 118 is mounted on the floor 
- within proximity of the. person. It may be. mounted on a. 
tripod, as shown, or fixed to the environment in some 
5 other way. Stationary scene camera 118 is aimed so that 
one or more .of the surfaces of interest (e.g. scene 
planes 120 and 122) are visible in the camera video 
image. Scene planes 120 and 122 mky be instruments on a 
control panel, visual presentations, or any other regions 

10 of visual interest. When the user looks forward, scene 
planes 12 0 and .122 are within the field of view of both 
the person and head-mounted scene camera 116. 

Eye tracker sensor and optics unit. 114 which 
incorporates the off-axis light source (e.g. the ring 

15 light source) produces a video image that is preprocessed 
and digitized by an eye tracker electronics unit 124 and 
sent to a computer 126. Computer 12 6, which is 
programmed appropriately, uses the resulting digital 
information from unit 124 to determine the relative 

20 locations of the pupil center and the reflection of the 
near infrared light source on the front surface of the 
cornea. From the pupil-to-corneal reflection vector, 
computer 126 determines the pointing direction of the eye 
with respect to the head mounted optics. The pointing 

25 direction of the eye is represented by two coordinates in 
computer memory that are proportional to eye azimuth and 
eye elevation angle, respectively (or the equivalent).. 

In the described embodiment, eye tracker 
electronics unit 124 and computer 126 are both part of a 

30 eyerhead tracker processor 128, which may be commercially 
obtained from Applied Science Group, Inc. of Walt ham,- MA-. • 
and is identified as the ASL model 4100H-EHS eye-head 
tracker system.. 

The pupil to corneal reflection technique for 

35 measuring eye pointing direction is described in the 
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literature, and is well-known to those skilled in the 
art. (See, for example, Young and Sheena, Methods & 
- Designs, Sunrvey.of eye movement recording methods, 

Behavior Research Methods and Instrumentation 1975, Vol. 
5 7(5), 397-492; Merchant & Morrisette, Remote measurement 
of eye direction allowing subject motion over one cubic 
foot of space, IEEE Transactions on Biomedical . 
Engineering, 1974, BME-21, 309-317; and Borah, "Helmet 
Mounted Eye Tracking for. Virtual Panoramic Display 
10 Systems", AAMRL-TR- 89-019 , Harry B. Armstrong Aerospace 
Medical Research Laboratory, Human Systems Division, Air 
Force Systems Command, Wright -Patterson AFB, August 
1989.) 

Of course, there are also other techniques for 
15 determining eye pointing direction with respect to the 

subject's head and which can be used to place azimuth and 
elevation eye angle coordinates (or the equivalent) in 
computer memory. Such techniques are also described in 
the literature. (See, e.g. Young & Sheena, as referenced 
2 0 above; and Borah, as referenced above) : 

Computer .12 6 maps the resulting coordinate values 
to a different set of coordinates which represent a 
horizontal and vertical location on the video image from 
head mounted scene camera 116. Note that since head 
25 mounted scene camera 16 moves along with the head and is 
optically located at the same (or nearly the same) 
.. position as the eye, it remains part of the same 

reference frame as the eye position detection system. In 
other words, there is a unique relation between eye 
30 pointing direction with respect to the- head, and point of 
gaze with respect to the scene camera image. There are 
several mapping techniques for mapping eye azimuth and 
elevation values to such a scene camera image field, 
including interpolation techniques and curve fit 
35 techniques. These techniques are described in the 
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literature, and are well-known to those practiced in the 
art. The technique used in the preferred embodiment is a 
curve fit technique (see, e.g. Sheena & Borah, 
"Compensation for Some Second order. Effects to Improve 
5- Eye Position Measurements", for D.F. Fisher, R.A. Monty, 
and J.W. Sanders (Eds) : Eve Movements: Cognition and 
Visual Perception , L. Erlbaum Assoc., 1981). 

The position coordinates of point of gaze with 
respect to the head mounted scene camera image, as 
10 described in the previous .paragraph, can be displayed on. 
a scene monitor 130 as a cursor, cross hairs, or other 
indicator, superimposed on the video image, from head- 
mounted scene camera 116. The ASL model 4100H-EHS 
includes the capability for such cursor superimposit ion . 
v 15 In addition, many other commercially available devices 
exist which superimpose cursors, cross hairs, or other 
symbols on a video signal at specified locations, whose 
coordinates are available in computer memory, and such 
devices can be used. 

2 0 Note that use of head -mounted scene camera 116 is 

. not a necessary part of the system but is described 
primarily because it is. readily available and is a common 
part of some eye tracking systems. 

The eye tracking system in Fig. 3 also includes a 

25 head tracker which determines the position, and 

orientation of the person's head. In the described 
embodiment, the head tracker is a device based on 
magnetic principles, such as the 3Space Tracker available 
from Polhemus, a Kaiser Aerospace & Electronics Company, 

30 or The Bird™ available from Ascension Technology, Inc. 
Other possible embodiments could utilize mechanical 
goniometers; ultrasonic devices, such as one offered 
commercially by Logitech, Inc.; optical devices; or any 
other device that can be used to measure head position 

3 5 and orientation. 
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The magnetic head tracker (MHT) shown, in Fig. 3 
includes an MHT sensor 132, an MHT transmitter 134, and 
. an MHT control unit 136. MHT sensor 132 is fastened to 
the subjects helmet, and MHT transmitter 134 is fixed to 
5 the environment near the subjects head. MHT control unit 
136 determines the position of MHT sensor 132 with 
respect to MHT transmitter 134 in 6 degrees of freedom 
and communicates, this information to computer 126 via an 
RS-232 interface. 
10 A program. in computer 126 uses information from 

the head tracker,, information about eye line of gaze with 
respect to the head (computed as described above) , and 
stored information about the location of surfaces in the 
environment (such as scene plane 120 and scene plane 
15. 122), to determine the location and direction of the eye 
line of gaze vector with respect . to the environment, the 
surface intersected by the line of gaze vector, and the 
location of the intersection point (point P in. Fig. 3) 
with respect to the surface intersected. In the 
20 illustrated embodiment, this data field, which is 

identified as "RS-232 data output", includes the number 
of the scene plane being viewed, and the horizontal and 
vertical coordinates of point of gaze on that surface 
(with respect to a coordinate frame predefined on that 
25 surface) . The data can be read by an external device on 
a standard RS-232 serial data port. A hew data field is 
available at the same update rate as that being used by 
the camera imaging the subject's eye. This is generally 
.60 times per second in the USA, when the eye tracker 
30 employs standard NTSC video format cameras, or 50 times 
per second in Europe. or other countries , . when the eye 
tracker employs cameras with standard European PAL video 
format . ■ • 

Computer 126. uses the point of gaze information to 
3 5 determine the location of gaze within the viewed scene as 
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shown by the video scene monitor and it superimposes a 
cursor or cross hairs on the image displayed on video 
scene monitor 130. The system can either use an 
appropriate set of transformations to map the point of 
5 gaze onto the scene image or it can be done by first 
calibrating to establish a reference point on the scene 
image. This latter approach involves having the subject 
look at a fixed reference point in the image scene to 
determine a reference line-of -gaze direction associated 
10 with that point. Then any changes in the line-of -gaze 
can be readily translated into an appropriate change in 
the point of gaze in the image scene. 

An alternate embodiment of the stationary scene 
camera implementation is shown in Fig. 4. In this 
r . 15 embodiment, the standard RS-232 data output available 

from eye-head tracker processor 128 (e.g. ASL model 4100- 
EHS) is read by an external PC-AT type computer .140. 
External computer 140 is equipped with an NTSC/VGA 
conversion board 142 such as the Redlake model NTSC 100 
• 20 Video Digitizer and VGA Overlay Controller. This 

commercially available board allows computer 14 0 to 
display an image from a standard NTSC format video camera 
118 on a computer VGA screen 144, and it also allows 
computer 140 to superimpose VGA graphics on this image. 
25 Computer 140 also includes a mouse 146 (or other pointing 
. device) that enables the user to move the cursor about on 
the video image and it includes programming capable of 
capturing and recording in memory the VGA coordinates of 
the cursor at the location at which the mouse is clicked. 
3 0 In the . embodiment shown by Fig. 4, the stationary 

scene camera video, image is input to NTSC/VGA conversion 
board 142 in external computer 140. Using information 
from the eye head tracker, a program in the external 
. computer superimposes a cursor, cross hairs, or other 
35 indicator showing the subject's point of gaze, on the VGA 
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image from stationary scene camera 118. 

It should be noted that the Sony camera which was 
described above is actually more appropriate used in a 
system in which the eye tracker camera is mounted 
5 stationary with respect to the floor. In essence, that 
means moving the head mounted cameras shown in Figs. 3 
and 4 to a floor mount system. Such a system would in 
principle operate the same way though some minor 
modifications would be necessary, esepcially in the 
0 software, due to the fact the frame of reference is now 
the room rather than the subject' s head. 

The invention is meant to cover all of the above- 
mentioned alternative approaches as well as others not 
specifically mentioned. The above-mentioned embodiments 
5 and others are within the following claims. 
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Claims : 

- What is claimed is : 

.1. A camera assembly for use in an eye tracking 
apparatus, said camera comprising: 
\5 a camera with a lens having an image axis; and 

a ring shaped light source disposed around the 
image axis and near the periphery of the lens aperture, 
said light source oriented to direct light along the 
camera axis toward the target. ■ 

10 m 2.- An eye line-of-gaze measurement apparatus 

comprising: 

an electronic camera with a lens having an image 

axis; 

a ring shaped light source disposed around the 
15 image axis and near the periphery of the lens aperture, 
. said light source, oriented to direct light along the 
camera axis toward a target; and 

a digital processor programmed to determine an eye 
line-of-gaze from the image of a retro-reflection 
20 obtained from the subject's eye. 

3. The apparatus of claim 2 wherein the ring- 
shaped light, source comprises an array of lights arranged 
in a circle to form a ring. 

4. The apparatus of claim 3 wherein the plurality 
25 of light sources. are evenly spaced about the circle. 

5. The apparatus of claim 4 wherein the light 
sources are LED ' s . 

6. A method, of generating a retro-reflection from 
a subject's eye for use in a line-of-gaze measurement 
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system that utilizes a bright pupil detection technique, 
said method comprising: 

producing an image of the subject's eye by using a 
camera that is characterized by a viewing axis; and 
5 illuminating the subject's eye with an off-axis 

illumination to produce a retro-reflection from the 
retina of the subject's eye. 

7 . The method of claim 6 wherein the step of 
illuminating the subject's eye comprises illuminating 
10 with a light source that is distributed around the 
viewing axis. 
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